Abstract: The surface defects of industrial products have an important impact on the aesthetic and performance. In order to realize the automatic detection and classification of surface defects of hot-rolled aluminum alloy sheets, a machine vision detection method was proposed based on photometric feature extraction and support vector machine classification. Aiming at five common defects such as indentation, scratches, stains, bruises, and pitting of aluminum alloy plates, corresponding image processing algorithms were designed according to their photometric stereoscopic characteristics. The defect area was reliably detected and the geometric shape and HSV feature of the defect were extracted. The obtained feature parameters were used as the input of the support vector machine, and the corresponding classifier was designed. The results show that the designed SVM classifier can be accurately detected by the type of surface defect, with the recognition rate of 96.5%.